11 research outputs found
On Positional and Structural Node Features for Graph Neural Networks on Non-attributed Graphs
Graph neural networks (GNNs) have been widely used in various graph-related
problems such as node classification and graph classification, where the
superior performance is mainly established when natural node features are
available. However, it is not well understood how GNNs work without natural
node features, especially regarding the various ways to construct artificial
ones. In this paper, we point out the two types of artificial node
features,i.e., positional and structural node features, and provide insights on
why each of them is more appropriate for certain tasks,i.e., positional node
classification, structural node classification, and graph classification.
Extensive experimental results on 10 benchmark datasets validate our insights,
thus leading to a practical guideline on the choices between different
artificial node features for GNNs on non-attributed graphs. The code is
available at https://github.com/zjzijielu/gnn-exp/.Comment: This paper has been accepted to the Sixth International Workshop on
Deep Learning on Graphs (DLG-KDD'21) (co-located with KDD'21
Joint Embedding of Structural and Functional Brain Networks with Graph Neural Networks for Mental Illness Diagnosis
Multimodal brain networks characterize complex connectivities among different
brain regions from both structural and functional aspects and provide a new
means for mental disease analysis. Recently, Graph Neural Networks (GNNs) have
become a de facto model for analyzing graph-structured data. However, how to
employ GNNs to extract effective representations from brain networks in
multiple modalities remains rarely explored. Moreover, as brain networks
provide no initial node features, how to design informative node attributes and
leverage edge weights for GNNs to learn is left unsolved. To this end, we
develop a novel multiview GNN for multimodal brain networks. In particular, we
regard each modality as a view for brain networks and employ contrastive
learning for multimodal fusion. Then, we propose a GNN model which takes
advantage of the message passing scheme by propagating messages based on degree
statistics and brain region connectivities. Extensive experiments on two
real-world disease datasets (HIV and Bipolar) demonstrate the effectiveness of
our proposed method over state-of-the-art baselines.Comment: Accepted to ICML 2021 Workshop on Computational Approaches to Mental
Healt
Dynamic Brain Transformer with Multi-level Attention for Functional Brain Network Analysis
Recent neuroimaging studies have highlighted the importance of
network-centric brain analysis, particularly with functional magnetic resonance
imaging. The emergence of Deep Neural Networks has fostered a substantial
interest in predicting clinical outcomes and categorizing individuals based on
brain networks. However, the conventional approach involving static brain
network analysis offers limited potential in capturing the dynamism of brain
function. Although recent studies have attempted to harness dynamic brain
networks, their high dimensionality and complexity present substantial
challenges. This paper proposes a novel methodology, Dynamic bRAin Transformer
(DART), which combines static and dynamic brain networks for more effective and
nuanced brain function analysis. Our model uses the static brain network as a
baseline, integrating dynamic brain networks to enhance performance against
traditional methods. We innovatively employ attention mechanisms, enhancing
model explainability and exploiting the dynamic brain network's temporal
variations. The proposed approach offers a robust solution to the low
signal-to-noise ratio of blood-oxygen-level-dependent signals, a recurring
issue in direct DNN modeling. It also provides valuable insights into which
brain circuits or dynamic networks contribute more to final predictions. As
such, DRAT shows a promising direction in neuroimaging studies, contributing to
the comprehensive understanding of brain organization and the role of neural
circuits.Comment: Accepted to IEEE BHI 202
Interpretable Graph Neural Networks for Connectome-Based Brain Disorder Analysis
Human brains lie at the core of complex neurobiological systems, where the
neurons, circuits, and subsystems interact in enigmatic ways. Understanding the
structural and functional mechanisms of the brain has long been an intriguing
pursuit for neuroscience research and clinical disorder therapy. Mapping the
connections of the human brain as a network is one of the most pervasive
paradigms in neuroscience. Graph Neural Networks (GNNs) have recently emerged
as a potential method for modeling complex network data. Deep models, on the
other hand, have low interpretability, which prevents their usage in
decision-critical contexts like healthcare. To bridge this gap, we propose an
interpretable framework to analyze disorder-specific Regions of Interest (ROIs)
and prominent connections. The proposed framework consists of two modules: a
brain-network-oriented backbone model for disease prediction and a globally
shared explanation generator that highlights disorder-specific biomarkers
including salient ROIs and important connections. We conduct experiments on
three real-world datasets of brain disorders. The results verify that our
framework can obtain outstanding performance and also identify meaningful
biomarkers. All code for this work is available at
https://github.com/HennyJie/IBGNN.git.Comment: Previous version presented at icml-imlh 2021 (no proceedings,
archived at 2107.05097), this version is accepted to miccai 202
PV2TEA: Patching Visual Modality to Textual-Established Information Extraction
Information extraction, e.g., attribute value extraction, has been
extensively studied and formulated based only on text. However, many attributes
can benefit from image-based extraction, like color, shape, pattern, among
others. The visual modality has long been underutilized, mainly due to
multimodal annotation difficulty. In this paper, we aim to patch the visual
modality to the textual-established attribute information extractor. The
cross-modality integration faces several unique challenges: (C1) images and
textual descriptions are loosely paired intra-sample and inter-samples; (C2)
images usually contain rich backgrounds that can mislead the prediction; (C3)
weakly supervised labels from textual-established extractors are biased for
multimodal training. We present PV2TEA, an encoder-decoder architecture
equipped with three bias reduction schemes: (S1) Augmented label-smoothed
contrast to improve the cross-modality alignment for loosely-paired image and
text; (S2) Attention-pruning that adaptively distinguishes the visual
foreground; (S3) Two-level neighborhood regularization that mitigates the label
textual bias via reliability estimation. Empirical results on real-world
e-Commerce datasets demonstrate up to 11.74% absolute (20.97% relatively) F1
increase over unimodal baselines.Comment: ACL 2023 Finding
Neighborhood-Regularized Self-Training for Learning with Few Labels
Training deep neural networks (DNNs) with limited supervision has been a
popular research topic as it can significantly alleviate the annotation burden.
Self-training has been successfully applied in semi-supervised learning tasks,
but one drawback of self-training is that it is vulnerable to the label noise
from incorrect pseudo labels. Inspired by the fact that samples with similar
labels tend to share similar representations, we develop a neighborhood-based
sample selection approach to tackle the issue of noisy pseudo labels. We
further stabilize self-training via aggregating the predictions from different
rounds during sample selection. Experiments on eight tasks show that our
proposed method outperforms the strongest self-training baseline with 1.83% and
2.51% performance gain for text and graph datasets on average. Our further
analysis demonstrates that our proposed data selection strategy reduces the
noise of pseudo labels by 36.8% and saves 57.3% of the time when compared with
the best baseline. Our code and appendices will be uploaded to
https://github.com/ritaranx/NeST.Comment: Accepted to AAAI 202
Open Visual Knowledge Extraction via Relation-Oriented Multimodality Model Prompting
Images contain rich relational knowledge that can help machines understand
the world. Existing methods on visual knowledge extraction often rely on the
pre-defined format (e.g., sub-verb-obj tuples) or vocabulary (e.g., relation
types), restricting the expressiveness of the extracted knowledge. In this
work, we take a first exploration to a new paradigm of open visual knowledge
extraction. To achieve this, we present OpenVik which consists of an open
relational region detector to detect regions potentially containing relational
knowledge and a visual knowledge generator that generates format-free knowledge
by prompting the large multimodality model with the detected region of
interest. We also explore two data enhancement techniques for diversifying the
generated format-free visual knowledge. Extensive knowledge quality evaluations
highlight the correctness and uniqueness of the extracted open visual knowledge
by OpenVik. Moreover, integrating our extracted knowledge across various visual
reasoning applications shows consistent improvements, indicating the real-world
applicability of OpenVik.Comment: Accepted to NeurIPS 202
Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey
Large language models (LLMs) have significantly advanced the field of natural
language processing (NLP), providing a highly useful, task-agnostic foundation
for a wide range of applications. However, directly applying LLMs to solve
sophisticated problems in specific domains meets many hurdles, caused by the
heterogeneity of domain data, the sophistication of domain knowledge, the
uniqueness of domain objectives, and the diversity of the constraints (e.g.,
various social norms, cultural conformity, religious beliefs, and ethical
standards in the domain applications). Domain specification techniques are key
to make large language models disruptive in many applications. Specifically, to
solve these hurdles, there has been a notable increase in research and
practices conducted in recent years on the domain specialization of LLMs. This
emerging field of study, with its substantial potential for impact,
necessitates a comprehensive and systematic review to better summarize and
guide ongoing work in this area. In this article, we present a comprehensive
survey on domain specification techniques for large language models, an
emerging direction critical for large language model applications. First, we
propose a systematic taxonomy that categorizes the LLM domain-specialization
techniques based on the accessibility to LLMs and summarizes the framework for
all the subcategories as well as their relations and differences to each other.
Second, we present an extensive taxonomy of critical application domains that
can benefit dramatically from specialized LLMs, discussing their practical
significance and open challenges. Last, we offer our insights into the current
research status and future trends in this area
FBNETGEN: Task-aware GNN-based fMRI Analysis via Functional Brain Network Generation
Functional magnetic resonance imaging (fMRI) is one of the most common
imaging modalities to investigate brain functions. Recent studies in
neuroscience stress the great potential of functional brain networks
constructed from fMRI data for clinical predictions. Traditional functional
brain networks, however, are noisy and unaware of downstream prediction tasks,
while also incompatible with the deep graph neural network (GNN) models. In
order to fully unleash the power of GNNs in network-based fMRI analysis, we
develop FBNETGEN, a task-aware and interpretable fMRI analysis framework via
deep brain network generation. In particular, we formulate (1) prominent region
of interest (ROI) features extraction, (2) brain networks generation, and (3)
clinical predictions with GNNs, in an end-to-end trainable model under the
guidance of particular prediction tasks. Along with the process, the key novel
component is the graph generator which learns to transform raw time-series
features into task-oriented brain networks. Our learnable graphs also provide
unique interpretations by highlighting prediction-related brain regions.
Comprehensive experiments on two datasets, i.e., the recently released and
currently largest publicly available fMRI dataset Adolescent Brain Cognitive
Development (ABCD), and the widely-used fMRI dataset PNC, prove the superior
effectiveness and interpretability of FBNETGEN. The implementation is available
at https://github.com/Wayfear/FBNETGEN.Comment: This paper has been accepted for presentation in MIDL 202